Machine Learning Python Programming Training

Machine Learning Types and Algorithms

Find the right job for right jobs/problems image.png

Machine Learning Types

Supervised

Unsupervised

Supervised Machine Learning

Classification

Support Vector Machine (SVM)

image.png

image.png

How SVM algorithm finds the best hyper-plane f(x)

  1. Calculate Loss value using loss function (hinge)
    • Hinge loss - C(x, y, f(x))=
      • 0 if y*f(x) >= 1 (Mostly, correct classification)
      • 1 – yf(x) if yf(x) < 1 (mis-classification)
    • If f(x) = W@x, C = 1 – y*(W@x).
    • The cost is 0 if the predicted value and the actual value are of the same sign. If they are not, the loss value will be calculated.
  2. Gradient Descents – to take partial derivatives on hinge loss with W
    • dC/dW = 0 if yi * (W@x) >= 1
          -yi *xi    if yi * (W@x) < 1 
  3. Update the parameter of f(x)
    • If no mis-classification, there is no update on w
    • If there is mis-classifcation, w = w - alpha (-yi xi )

image.png

Pros:

Cons:

Why Python for Machine Learning

Python is most popular / effective programming language for Machine Learning

Popular ML packages

Iris data description

image.png

Iris Data label: 0 = 'Setosa', 1 = 'Vesicolor', 2 = 'Virginica'

image.png

KNN (K Nearest Neighbors)

How KNN works?

  1. Calculate Distance between new data and training data
  2. Find the class of closest neighbors (how many A and B in a given k)
  3. Vote for labels which has most votes in a given k

image.png

image.png

Pros:

Cons:

Decision Trees

The widely used machine learning algorithm that use tree to visually and explicitly represent decision and its conditions. image.png

How Decision Tree works

  1. Pick each feature (one dimension of X input data) – X1, X2, X3
  2. For that feature, pick the condition : X1> 4
  3. Calculate the impurity of that condition of that feature
  4. Go through 2 to 3 for all the possibility of that feature and pick the best lowest impurity (feature, condition)
  5. Pick another feature – X1, X2, X3
  6. Go through 2 to 4 to find the lowest impurity.
  7. Pick the lowest impurity from all the feature
  8. Go to the next leaf
from sklearn.tree.export import export_text tree_rules = export_text(d_tree, feature_names=iris['feature_names']) print(tree_rules)from sklearn import tree import pydotplus import collections from IPython.display import Image import graphviz # Create DOT data dot_data = tree.export_graphviz(d_tree, out_file=None, feature_names=iris['feature_names'], class_names=iris['target_names']) # Draw graph graph = pydotplus.graph_from_dot_data(dot_data) # Show graph #Image(graph.create_png())

Random Forests

The ensemble of decision trees that uses multiple decision trees.

Steps

  1. Randomly choose bootstrap sample of size n (k samples from the training data).
  2. Grow decision tree from bootstrap sample (sampling random sets of observations with replacement).
  3. Repeat 1 and 2 on k times
  4. Aggregate the prediction by each tree and choose the predictor with the majority vote. (e.g., Class A from tree1, Class B from tree 3 …)

Why Random Forest? Can build more robust model that has a better generalization performance and less subject to overfitting.

image.png

XGBoost

Optimized Gradient Boostig tree-based algorithm that

Note : Random Forest model is a bagging ensemble model while XGBoost is a sequential ensemble model.

image.png

Logistic Regression

image.png

Logistic Regression Architecture

Hypothesis function : hθ(x) = 1 / (1 + e-z) where z= ( θ0x0 + θ1x1 + θ2x2 + θ3x3 +,, + θnxn )

image.png

Regression

image.png

Unsupervised Machine Learning

k-means Clustering

How it works

  1. Randomly pick the location of k(given/assigned) centroids as initial cluster centers.
  2. Assign each sample to the nearest centroid for k cluster.
  3. Move the centroids to the center of the samples (clusters) that were assigned to it.
  4. Repeat 2 and 3 until the cluster assignments do not change under a tolerance.

image.png

Artificial Neural Network

image.png

Human Neural Network vs Artificial Neural Network (ANN)

Human Neurons : Neural Network – 100 billions image.png

Artificial Neurons : Artificial Neural Networks image.png

Artificial Neuron

image.png

What does “Artificial Neuron” learn?

Machine Learning – To learn parameters

How “Artificial Neuron” learns

Cost function of ANN

image.png

Python Codes for Deep Neural Network

DNN architecture design

image.png

Activation Functions

Type

Linear function

Sigmoid or Logistic function

Tanh or Hyperbolic Tangent function

ReLU (Rectified Linear Unit) function

Softmax function

Examples of Activation Function

Python codes for Activation Function model = Sequential() model.add(Dense(4, input_dim=3, activation='relu')) model.add(Dense(2, activation='relu')) model.add(Dense(2, activation='softmax'))

Loss Function in Machine Learning

Regressive Loss Function

Mean Square Error(MSE)

the average of the squared differences between the predicted and actual value image.png

Mean Absolute Error(MAE)

the sum of absolute differences between our target and predicted variables image.png

Python code Example for regressive loss function :

model.compile(loss='mse', optimizer=Adam(), metrics=['accuracy' ])

Classification Loss Function

For Classification target variables

Binary classification

Multi classification

Optimizer in Machine Learning

Optimizer types image.png

Python codes examples for optimizer

Deep Neural Network (DNN) / Deep Learning

image.png

Tensorflow DNN Playground

http://playground.tensorflow.org image.png

CONCLUSION

CONTACT INFORMATION

Your comments and questions are valued and encouraged. Please contact at:
Kevin Lee
AVP of Machine Learning and AI Consultant
Kevin.lee@genpact.com
Tweet: @HelloKevinLee
LinkedIn: www.linkedin.com/in/HelloKevinLee/